10 research outputs found
Statistical Inference on Stochastic Graphs
This thesis considers modelling and applications of random graph processes.
A brief review on contemporary random graph models and a general Birth-Death
model with relevant maximum likelihood inference procedure are provided in chapter
one. The main result in this thesis is the construction of an epidemic model by
embedding a competing hazard model within a stochastic graph process (chapter
2). This model includes both individual characteristics and the population connectivity
pattern in analyzing the infection propagation. The dynamic outdegrees and
indegrees, estimated by the model, provide insight into important epidemiological
concepts such as the reproductive number. A dynamic reproductive number based
on the disease graph process is developed and applied in several simulated and actual
epidemic outbreaks. In addition, graph-based statistical measures are proposed
to quantify the effect of individual characteristics on the disease propagation. The
epidemic model is applied to two real outbreaks: the 2001 foot-and-mouth epidemic
in the United Kingdom (chapter 3) and the 1861 measles outbreak in Hagelloch,
Germany (chapter 4). Both applications provide valuable insight into the behaviour
of infectious disease propagation with di erent connectivity patterns and human
interventions
VCD: A Video Conferencing Dataset for Video Compression
Commonly used datasets for evaluating video codecs are all very high quality
and not representative of video typically used in video conferencing scenarios.
We present the Video Conferencing Dataset (VCD) for evaluating video codecs for
real-time communication, the first such dataset focused on video conferencing.
VCD includes a wide variety of camera qualities and spatial and temporal
information. It includes both desktop and mobile scenarios and two types of
video background processing. We report the compression efficiency of H.264,
H.265, H.266, and AV1 in low-delay settings on VCD and compare it with the
non-video conferencing datasets UVC, MLC-JVC, and HEVC. The results show the
source quality and the scenarios have a significant effect on the compression
efficiency of all the codecs. VCD enables the evaluation and tuning of codecs
for this important scenario. The VCD is publicly available as an open-source
dataset at https://github.com/microsoft/VCD
Trustworthy Experimentation Under Telemetry Loss
Failure to accurately measure the outcomes of an experiment can lead to bias
and incorrect conclusions. Online controlled experiments (aka AB tests) are
increasingly being used to make decisions to improve websites as well as mobile
and desktop applications. We argue that loss of telemetry data (during upload
or post-processing) can skew the results of experiments, leading to loss of
statistical power and inaccurate or erroneous conclusions. By systematically
investigating the causes of telemetry loss, we argue that it is not practical
to entirely eliminate it. Consequently, experimentation systems need to be
robust to its effects. Furthermore, we note that it is nontrivial to measure
the absolute level of telemetry loss in an experimentation system. In this
paper, we take a top-down approach towards solving this problem. We motivate
the impact of loss qualitatively using experiments in real applications
deployed at scale, and formalize the problem by presenting a theoretical
breakdown of the bias introduced by loss. Based on this foundation, we present
a general framework for quantitatively evaluating the impact of telemetry loss,
and present two solutions to measure the absolute levels of loss. This
framework is used by well-known applications at Microsoft, with millions of
users and billions of sessions. These general principles can be adopted by any
application to improve the overall trustworthiness of experimentation and
data-driven decision making.Comment: Proceedings of the 27th ACM International Conference on Information
and Knowledge Management, October 201
Meeting effectiveness and inclusiveness: large-scale measurement, identification of key features, and prediction in real-world remote meetings
Workplace meetings are vital to organizational collaboration, yet relatively
little progress has been made toward measuring meeting effectiveness and
inclusiveness at scale. The recent rise in remote and hybrid meetings
represents an opportunity to do so via computer-mediated communication (CMC)
systems. Here, we share the results of an effective and inclusive meetings
survey embedded within a CMC system in a diverse set of companies and
organizations. We correlate the survey results with objective metrics available
from the CMC system to identify the generalizable attributes that characterize
perceived effectiveness and inclusiveness in meetings. Additionally, we explore
a predictive model of meeting effectiveness and inclusiveness based solely on
objective meeting attributes. Lastly, we show challenges and discuss solutions
around the subjective measurement of meeting experiences. To our knowledge,
this is the largest data-driven study conducted after the pandemic peak to
measure, understand, and predict effectiveness and inclusiveness in real-world
meetings at an organizational scale
Analysis of Problem Tokens to Rank Factors Impacting Quality in VoIP Applications
User-perceived quality-of-experience (QoE) in internet telephony systems is
commonly evaluated using subjective ratings computed as a Mean Opinion Score
(MOS). In such systems, while user MOS can be tracked on an ongoing basis, it
does not give insight into which factors of a call induced any perceived
degradation in QoE -- it does not tell us what caused a user to have a
sub-optimal experience. For effective planning of product improvements, we are
interested in understanding the impact of each of these degrading factors,
allowing the estimation of the return (i.e., the improvement in user QoE) for a
given investment. To obtain such insights, we advocate the use of an
end-of-call "problem token questionnaire" (PTQ) which probes the user about
common call quality issues (e.g., distorted audio or frozen video) which they
may have experienced. In this paper, we show the efficacy of this questionnaire
using data gathered from over 700,000 end-of-call surveys gathered from Skype
(a large commercial VoIP application). We present a method to rank call quality
and reliability issues and address the challenge of isolating independent
factors impacting the QoE. Finally, we present representative examples of how
these problem tokens have proven to be useful in practice
Improving Meeting Inclusiveness using Speech Interruption Analysis
Meetings are a pervasive method of communication within all types of
companies and organizations, and using remote collaboration systems to conduct
meetings has increased dramatically since the COVID-19 pandemic. However, not
all meetings are inclusive, especially in terms of the participation rates
among attendees. In a recent large-scale survey conducted at Microsoft, the top
suggestion given by meeting participants for improving inclusiveness is to
improve the ability of remote participants to interrupt and acquire the floor
during meetings. We show that the use of the virtual raise hand (VRH) feature
can lead to an increase in predicted meeting inclusiveness at Microsoft. One
challenge is that VRH is used in less than 1% of all meetings. In order to
drive adoption of its usage to improve inclusiveness (and participation), we
present a machine learning-based system that predicts when a meeting
participant attempts to obtain the floor, but fails to interrupt (termed a
`failed interruption'). This prediction can be used to nudge the user to raise
their virtual hand within the meeting. We believe this is the first failed
speech interruption detector, and the performance on a realistic test set has
an area under curve (AUC) of 0.95 with a true positive rate (TPR) of 50% at a
false positive rate (FPR) of <1%. To our knowledge, this is also the first
dataset of interruption categories (including the failed interruption category)
for remote meetings. Finally, we believe this is the first such system designed
to improve meeting inclusiveness through speech interruption analysis and
active intervention
Full Reference Video Quality Assessment for Machine Learning-Based Video Codecs
Machine learning-based video codecs have made significant progress in the
past few years. A critical area in the development of ML-based video codecs is
an accurate evaluation metric that does not require an expensive and slow
subjective test. We show that existing evaluation metrics that were designed
and trained on DSP-based video codecs are not highly correlated to subjective
opinion when used with ML video codecs due to the video artifacts being quite
different between ML and video codecs. We provide a new dataset of ML video
codec videos that have been accurately labeled for quality. We also propose a
new full reference video quality assessment (FRVQA) model that achieves a
Pearson Correlation Coefficient (PCC) of 0.99 and a Spearman's Rank Correlation
Coefficient (SRCC) of 0.99 at the model level. We make the dataset and FRVQA
model open source to help accelerate research in ML video codecs, and so that
others can further improve the FRVQA model
DNN No-Reference PSTN Speech Quality Prediction
Classic public switched telephone networks (PSTN) are often a black box for
VoIP network providers, as they have no access to performance indicators, such
as delay or packet loss. Only the degraded output speech signal can be used to
monitor the speech quality of these networks. However, the current
state-of-the-art speech quality models are not reliable enough to be used for
live monitoring. One of the reasons for this is that PSTN distortions can be
unique depending on the provider and country, which makes it difficult to train
a model that generalizes well for different PSTN networks. In this paper, we
present a new open-source PSTN speech quality test set with over 1000
crowdsourced real phone calls. Our proposed no-reference model outperforms the
full-reference POLQA and no-reference P.563 on the validation and test set.
Further, we analyzed the influence of file cropping on the perceived speech
quality and the influence of the number of ratings and training size on the
model accuracy